The Exercise Format Editor

An important condition for efficient multimedia learning is that the learner takes an active part in using the material to be learned. One possibility to initiate active learning and intensive information processing is to provide exercises of multiple complexity and difficulty. Hence, the question of how to support the systematic design and construction of multiple exercises for multimedia learning is of critical interest for instructional designers. This paper presents the psychological background and the application of the Exercise Format (EF) and the Exercise Format-Editor, a multimedia tool kit for the systematic construction of learning tasks. The development of this tool kit was based on psychological findings on task analysis and on self-regulated learning with multimedia learning environments. In order to support the construction of a large variety of exercises, EF (a) allows the integration of various learning contents, materials and media, (b) allows the preparation of different item forms, (c) allows an automatic assessment of the learner’s response, and (d) provides informative tutoring feedback types for typical mistakes. From a psychological viewpoint, an important prerequisite of efficient self-regulated learning is that the learners actively use the material to be learned. This entails gaining selected information on a particular subject, working through this information, analysing it, and relating this information to one's own knowledge. For these purposes learning tasks or exercises are of particular interest because they can initiate and support the active and intensive information processing. Numerous studies (for a review see Hamaker, 1986) reveal that working with exercises can facilitate (a) a longer retention and deeper understanding of the learning material, (b) the structuring and application of knowledge, and (c) the test of the acquired knowledge. Moreover, learning tasks in self-regulated learning via multimedia technologies play a prominent role in monitoring and managing the learning process, because correctly respective incorrectly solved tasks can provide a great deal of information (e.g. feedback whether the learning objectives are achieved, cues as to which concepts and principles are important to know, feedback indicating mistakes and thus helps to identify the learning material that should be repeated). However, useful information is only provided if learning tasks of different complexity and difficulty are constructed systematically with regard to the instructional objectives of the given learning context and with regard to learner characteristics (e.g. prior knowledge, motivation). In order to help instructional designers to construct learning tasks that meet these requirements we developed the Exercise Format (EF) on the basis of the current knowledge on task analyses (for a The Exercise Format Editor 3 detailed description see Jonassen, Tessmer & Hannum, 1999) and on selfregulated learning (c.f. Boekarts, 1996). Constructing learning tasks with regard to different learning objectives and different learner characteristics requires knowing which dimensions and/or attributes of a learning task can be varied systematically in order to design exercises of different complexity and difficulty. Hence, the purposes of this paper are (a) to give an overview of the dimensions and attributes that can be varied systematically when constructing learning tasks, and (b) to describe the functions and some applications of the EF-editor. Dimensions of (complex multimedia) learning tasks In general a task consists of two elements: the question or problem, and the answer respective solution. According to Klauer (1987), an exercise can be defined as the interconnection of a stimulus component and a response component. The stimulus component consists of the content, which is presented in a certain item form. The response component specifies what the learner should do with the stimulus component, e.g. to mark the correct alternative, to write down a short answer, or to write an essay in order to solve the task. Klauer (1984) mentioned that the item form together with the presented content determine the cognitive demands of a test item that a learner has to cope with. This definition is useful for the systematic construction of test items. However, we are especially interested in constructing learning tasks that, beside the test of the acquired knowledge, should promote and facilitate the knowledge and skill acquisition as well as the acquisition of meta-cognitive knowledge and skills. In contrast to test items, such learning tasks should offer the possibility to solve the task by providing informative feedback and multiple-try strategies (see Narciss & Huth, in this volume). Thus, the learner can solve the learning task in an interactive way. This interactive solving of a learning task entails not only a stimulus component and a response component, but also feedback loops as well as repeated working on one task, respectively subtask. Thus, we propose to describe (complex) learning tasks or exercises by at least three dimensions (see figure 1): First, each exercise is related to a specific part of a knowledge domain in addition to cognitive operations that are required for the task solution. These aspects – (a) the content, and (b) the cognitive operations both belong to the cognitive dimension in learning tasks. Second, in order to initiate and promote an active self-regulated learning the interactive dimension of learning tasks is of particular importance. This dimension consists of (a) a feedback component, and (b) an instructional tutoring component. The third dimension is the formal dimension of learning tasks and refers to (a) the item composition, and (b) the form and mode in which the content is presented. The Exercise Format Editor 4 Figure 1: Three dimensions of complex multimedia learning tasks and relevant steps for the systematic construction of learning tasks The cognitive dimension The cognitive dimension of an exercise is determined by (a) basic elements of the instructional content which are presented in the exercise (e.g. facts, events, rules), and (b) the cognitive operations (e.g. recognize, recall, transform, identify, classify) that are required within the exercise. The aim of a systematic task construction is to develop exercises that differ in their difficulty and complexity. Thus, it is necessary to take into consideration different levels of knowledge elements, just as different levels of cognitive operations for the item construction. The basis for the identification of relevant knowledge elements and cognitive skills is the detailed definition of learning objectives in terms of concrete learning outcomes. This specification indicates precisely what the learners have to do with the selected knowledge elements. “Applying a fact to solve a problem is a different instructional outcome from just stating a fact.” (Jonassen, Tesssmer & Hannum, 1999, p. 208) For identifying the relevant knowledge elements it is convenient to analyse and structure the subject matter using procedures of content analyses (for a detailed description see Jonassen et al. 1999, or Albert & Lukas, 1999). This entails breaking down the extensive content into smaller elements differing in their complexity, e.g. from simple facts across principles to particular theories. In the next step, relationships between and among the knowledge The Exercise Format Editor 5 elements have to be identified and depicted. Such relationships for example could be semantic, e.g. “y is condition to x”, “x is a generic term of y”, “x is prior knowledge to y”. These relationships for example can be used for sequencing exercises with several difficulties from the most simple to the most complex. In addition, the results of the cognitive task analyses can be useful for the determination of transparent evaluation criteria and the formulation of instructional tutoring information and feedback messages to help learners succeeding the task. The specification of the cognitive operations that can be applied to the selected knowledge elements of the subject matter can be based on a familiar taxonomy of learning objectives such as Bloom’s taxonomy of educational objectives (Bloom, Hastings, & Madaus, 1971). This cognitive taxonomy is divided into six major categories: (a) knowledge, (b) comprehension, (c) application, (d) analysis, (e) synthesis, and (f) evaluation. Each category is further subdivided into more specific objectives. Often the categories of these familiar taxonomies are still too general and must be broken down into more specific categories before being used for systematic item construction (Jonassen, et al. 1999). In the present approach for systematic exercise construction we propose to specify the relevant set of cognitive skills with regard to the learning objectives and the given domain of knowledge. This goal-related strategy of skill categorization provides the opportunity to determine and match the intended cognitive operations specifically for the given learning context. In an instructional context on operant conditioning the learning objectives for example might be to acquire the principles of reinforcement and apply them to the analysis of problematic situations in instructional contexts. The cognitive operations related to these objectives could, for example, be defined as (a) remembering, (b) transforming, (c) classifying, (d) arguing and/or concluding (Narciss & Proske, 2001). In general, it is possible to assign each knowledge element to each cognitive operation in order to construct a learning task. But in some cases the assignment is not suitable for all combinations. The decision which cognitive operations should be required for which knowledge element should be rather matched to the learning objectives defined before. Furthermore, exercises should be designed for each of the defined learning objectives. The interactive dimension As mentioned before, the learner-learning task interaction is of particular interest in multimedia settings. Improving the (meta-cognitive) knowledge when working with learning tasks is only possible if there is information available which supports the solving of the tasks. Without this interactive function of The Exercise Format Editor 6 learning tasks a self-regulation of the learning process is difficult or even hardly possible. The interactive dimension of learning tasks consists of (a) a feedback component and (b) an instructional tutoring component. For each of these components the content, function and modus in which the information will be provided has to be selected. These decisions depend, on the one hand, on learning objectives and on the other hand on expected sources of mistakes or difficulties for the given exercises. In order to identify sources of systematic mistakes several approaches of error analyses might be used. First, the identification of typical errors might be based on teachers’ or instructors’ experiences in the relevant field of knowledge. Second, an empirical item and error analysis can be used to detect systematic errors. Third, the results of tasks analyses might be used to determine potential sources of errors related to (a) the relevant knowledge elements, (b) the cognitive operations to proceed, and (c) the meta-cognitive knowledge and strategies that are required in order to solve the task. The feedback and the instructional tutoring information can refer to each of these aspects of the learning task (Narciss & Huth, in this volume). In computer based instruction it is particularly important to determine the way feedback will be presented to the learner. With regard to the instructional objectives and the cognitive requirements of the exercise there are different possibilities to provide feedback information, e.g. knowledge of result or knowledge of the correct result, as well as elaborated feedback. For the purpose of supporting the interactive solving of a complex learning task elaborated tutoring feedback forms are of major interest. In general, the content of a elaborated feedback message consists of two components: (a) an evaluative component that provides information on the learner’s performance level, and (b) a informational component that gives additional information related either to the content of the learning task, the strategy to succeed, or the error(s) that have been occurred. Such additional information could be, for example, hints on possible information sources, hints on meta-cognitive strategies, analogies, or examples (Narciss & Huth, in this volume). The function of elaborated tutoring feedback in interactive instruction is to support learners in mastering the requirements of the learning tasks and to initiate further learning activities. Thus, feedback should not immediately offer the correct response, but provide knowledge on how to proceed in combination with a multiple-try strategy (for detailed description of feedback design see Narciss & Huth, in this volume). Whereas feedback has to be presented after working on learning tasks or sub-tasks, instructional tutoring information can be provided during the whole instructional process. Thus, if the learner needs some more information for the task solution the possibility is given to look for helpful information. The content of the instructional information is similar to the informational component of the The Exercise Format Editor 7 feedback information, and could be related to the content of the exercise, to successful strategies for the task solution, or to possible sources of errors. In order to avoid a cognitive overload, this information should be provided in manageable pieces. The form and modus of the presentation of these tutoring pieces of information depends on the learning objectives and the cognitive task requirements. In multimedia settings, it is possible to give the learner the opportunity to actively get such instructional information, e.g. by clicking on a help button. But it is also possible to present this information automatically after a particular time period without learner entries.